Neural Networks for 2D to 3D Human Pose Estimation

Neural networks have nowadays been applied to almost every conceivable problem in computer vision. Human pose estimation is another long standing problem which is now being revisited thanks to this novel technique. This work investigates two questions repeatedly asked when applying neural networks: how to obtain enough data to train them, and more fundamentally how to leverage structure of the problem in the design of the appropriate network? In the context of human pose estimation we propose a method of data augmentation by adding noise in the joint angle values of the pose to address the first issue, and we use regression of joint angles followed by application of forward kinematics algorithm to obtain 3D positions of the joints, like [25], to address the latter. Gradients can be back-propagated through the forward kinematics and hence such layer can be used in the end-to-end training of the neural network. Such approach enforces some of the structural constraints of the human body rather than performing estimation of multiple independent 3D positions. We have found that data augmentation is highly beneficial for the performance of the system. We also found that forward kinematics layer does not further improve the performance, at least in our experimental setting.